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1. INTRODUCTION 

The collective behaviors of self-organized, independent and decentralized swarms, for example flocks 
of birds, schools of fishes, branches of organic plans and social insects with self-organization and division of 
labor as the core components are categorized as swarm intelligence (SI). Self-organization is characterized by 
low-level components interactions whereas the division of labor included simultaneous tasks perform by spe- 
cialized individuals in cooperation. Both of these fundamental concepts are essential and adequate properties to 
attain swarm intelligence behaviors [I]-4]. Artificial bee colony (ABC) algorithm is one of the newest swarm 
intelligence based algorithms which was proposed by Karaboga [5]. ABC algorithm simulates the foraging be- 
havior of honey bees. It is very direct, robust and population-based. In the ABC algorithm, the process begins 
with some initial solutions (population), which try to improve them toward optimal solutions by distributing 
the bees colony into two separate groups which are called employed and unemployed bees. 

There are three essential components of forage selection namely food sources, employed and unem- 
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ployed. Food sources are defined as the value function and depend on numerous factors (adjacency to the nest, 
aggregation of the energy, and the ease of extracting this energy). Employed bees are linked with a particular 
food source at which they are being occupied at. They bring with them vital information regarding particular 
sources such as the distance and direction from the nest, the profitability of the source, and then share this in- 
formation with a certain probability. Meanwhile, the unemployed bees are continuously searching and looking 
out for a food source to be exploited [5]. The number of food sources is directly proportional to the number of 
employed bees since for every food source there is only one employed bee. The computation of fitness value 
is dependent on the quality of the food source and is associated with its position. The process of searching for 
food source by the swarm of bees reflects the process involved in finding the optimal solution [6]. 

There are two imperative mechanisms in the ABC algorithm which are the exploitation and exploration 
process. The exploitation process occurs whenever the employed bees approach and work on the food sources. 
Once the employed bees determine the amounts of nectar of the food sources, the onlooker bees will proceed 
to the value of source that has the highest probability and then calculates the amount of nectar. The end of 
exploitation process is indicated by exhaustion of food sources. In the meantime, exploration process started 
when bee scouts are deployed to search and find new food sources randomly [7], [8]. The improvement and 
modifications to this algorithm have been determined is numerous literature such as [9J—[19]. Nevertheless, 
there are some inefficiencies occurred in terms of exploitation and convergence ability associated with ABC 
algorithm. Based on previous studies ABC algorithm performance is superior during exploration stage however, 
the performance is weaker during exploitation stage. At the biggest scale level, the exploration process is related 
to the ability of exploring and seeking for global optimum independently while the exploitation involves the 
distribution of the acquired knowledge to search for better solutions. This fact is the main goal of this research, 
where in the exploitation phase a quasi-Newton scheme is introduced so that it can increase the exploitability 
of the ABC algorithm. 

The rest of this paper is structured as follows. Section 2 introduces the problem formulation of the 
global optimization. Section 3 describes the original ABC algorithm. Section 4 presents the proposed mod- 
ification of ABC algorithm through manipulation of Davidon-Flecher-Powell (DFP) and Broyden-Flecher- 
Goldfarb-Shanno (BFGS) schematics, numerical results and discussion are discussed in section 5, and the 
conclusion of the work is summarized in section 6. 


2. PROBLEM FORMULATION 
Considering the problem of finding a global solution of, 


minzep f(z) (1) 


where f : D C R” — Ris a continuous function, and D is closed and bounded region. In order to obtain the 
global minimum point, any unconstrained algorithm should obtain a point x* such that, 


f(a") < fl), Yæ eD) (2) 


In recent literature many algorithms have been proposed to solve the problem (i), see for example [20]—(22]. 

However, this paper focused on tackling the particular difficult case of problem in which the evaluations 
of second derivatives are too complex and time consuming. In addition, the initial guess point to start the 
algorithm involves high level of difficulty to predict. Therefore, exploring and developing a more efficient 
method to resolve the problem (1) is a significant and valuable scientific research area. 


3. ARTIFICIAL BEES COLONY ALGORITHM 
The algorithm consists of four main parts [23]. The first part initialized the population of unemployed 
bees for exploration search and employed bees for the food source by using (3): 


ig = aP" + (ae — gmin), p= lpm sn, j=l, ,D (3) 


max min 


where «;; represents jth dimension of the ith employed bee. «/"** is the upper boundary, x}™™ is the lower 
boundary of the food source position in dimension jth respectively. œ represents the uniform random number 
within the interval [0, 1]. The maximum number of cycles (MNC) is utilized to control the number of iterations 
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and it also acts as a termination criterion. In the second part, the fitness value will entice the onlooker bees to 
execute the exploitation search. A new candidate solution will be generated for each employed bee. The initial 
solution in the first part is replicated to a new solution (v; = x;). Then, a uniform randomly chosen parameter 
j of the solution is revised by using (4): 


Vij = Tij + (Tij — Tki). (4) 


i,k € {1,---,n}, j © {1,---,D} and i Æ k, j is randomly chosen parameter. x; is a randomly chosen 
solution in the neighborhood of x; and ¢ is the uniform random number within the interval [0,1]. The fitness 
value of the solution can be determined by (5): 


Al if fi>0 
—) TR w Jiz 
i Ta if fi <0 ©) 


where f; is the objective value for ith candidate solution. If the current solution is superior than previous one 
then the candidate solution substitutes to the current solution and the abandonment is reset to zero, otherwise 
it will be incremented by one. In the third part, all information related to the nectar amount and respective 
position of food sources will be communicated by the employed bees to the onlooker bees. The onlookers 
will proceed to evaluates the nectar information from employed bees and choose the food source based on the 
highest probability which is dependent on the nectar amount by (6). 


fit; 
Pi = T F 
Xj fit; 


In the next step, the candidate solution is updated by using (4). If the current solution is better than 
the previous one then candidate solution substitutes the current solution and the abandonment is reset to zero, 
otherwise it will be incremented by one. In the fourth part, the employed bee will turn into a scout bee if the 
abandonment limit reached the predefined limit and look out for a new food source. Scout bee will then check 
all potential solutions for a predefined limits. The scout bee will generate the solution by using (3) and set the 
abandonment back to zero. Then scout bee turns into the employed bee and hence prevents stagnation of the 
algorithm. Based on the explanation above, the original ABC algorithm is given in algorithm 1. 


(6) 


Algorithm 1 Basic algorithm of ABC bees colony 


Data: Initilization: the population of solution x;; i = 1,--- ,n; 7 = 1,--- , D; trial = 0 is the non improvemnet number of the solution 
Zi, j used for abondonment, maximum cycle number, limit = A food source which could not be improved through limit trial and 
abandoned by its employed bee. 

Evaluate the population Set cycle = 1 repeat 

{Produce a new food source population for employed bees} for i=/ to n do 

Produce a new food source v; for employed bees of the food source x; by using and determine its quality Apply a greedy 

selection process between v; and x; and select the better one if x; does not improve then 
| trial = trial; + 1 Otherwise trial; + 1 = 0 


end 
Calculate the probability value p; by using (6) for the solutions using fitness values by using {Produce a new food source 
population for onlooker bees} Sett = 0,i = 0 repeat 
if random < p; then 
{Produce a new v; food source by using (4) for onlooker bees} Apply a greedy selection process between v; and x; and 
select the better one if Solution x; does not improve then 
| trial; = trial; + 1 Otherwise trial; +1 =0 t=t+1 


end 

until t =n 

{Determine scout} if max(trial;) > limit then 

Replace x; with a new randomly produces solution by using 


end 
Memorize the best solution achieved so far cycle = cycle + 1 


until {cycle = Maximum cycle Number} 
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4. ARTIFICAL BEES COLONY ALGORITHM ENHANCEMENT 

Many attempts have been made in the exploitation process to enhance the performance of the ABC 
algorithm. In this section, the exploitation part is enhanced by using a quasi-Newton algorithm. The algorithm 
does not need explicit expressions of second partial derivatives. This is normally denoted as variable metric 
methods [24-27]. As implied by the term, the basic concept of this method is the classical Newton’s method, 
whereby the new point is calculated. 


k41 = Tk — H; ge, (7) 


Where H is the Hessian matrix and g is the gradient of the objective function, respectively. The basic principle 
in the quasi-Newton algorithm is the search direction which is based on an n x n direction matrix B which 
supports the same objective as the inverse of the Hessian matrix in Newton’s method. This matrix is computed 
by using the first-order derivative of f(x) collected from the previous iterations. Moreover, as the number of 
iterations increased, B progressively achieving a more precise representation of the inverse of Hessian and for 
convex quadratic objective functions that become identically to the inverse of Hessian in (n + 1) iterations. The 
two most important schemes of quasi-Newton are DFP and BFGS schemes. They will be described in detail 
how it replaces the exploitation process in the ABC algorithm. 

Based on the theoretical concept of Newton’s method which locally approximates the problem 
by using the quadratic function. The minimum point of quadratic approximation is utilized as the starting 
point for the subsequent iterations. This brings to next Newton’s method iteration as given in (7p. Generally, 
if the initial point is not sufficiently close to the optimal solution, then this method does not has the descent 
property. Nevertheless, the quasi-Newton method guarantees that the algorithm processes the descent property 
by amending the iteration: 


Tk+1 = Tk — Ak BkIk, (8) 


where 
B= Ik for the steepest-descent algorithm (9) 
E \ H,! forthe Newton’s method 


and Bẹ is an n x n real matrix that approximate the Hessian matrix. During constructing the approximation to 
the inverse of Hessian, objective function and gradient will be used solely. The 6,41 computation is achieved 
by an incremental update to Bx in the following subsection. 


4.1. Davidon-Flecher-Powell scheme 

This scheme was developed by Davidon in 1959 and modified by Fletcher and Powell in 1963. The 
algorithm guarantees that Bẹ to be positive definite for all k. However, for the case of larger non-quadratic 
problems, the algorithm tends to get stuck in certain problematic situations. This phenomenon is associated 
with By becoming nearly singular [20]. The scheme of DFP is given, 


Updating the matrix Bẹ using (10): 
Bra = Be + Uk + Ck (10) 
where U;, and C;, are calculated by using 
Up n s sT i Ch _ —Zk-Z2k 
Sk-VUk Uk -Zk 


Sk = adk; Yk = Jk+1 — Gk 
and 
gk+1 = VE (2k41); ze = Bree. 
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4.1.1. The quasi-Newton with DFP scheme algorithm 

The quasi-Newton with DFP Scheme algorithm is recorded. The next subsection is describing the 
refinement of the ABC Algorithm by introducing the DFP Scheme. In this work, the DFP scheme is used in 
the exploitation phase of the ABC algorithm. By doing this, the convergence of the original ABC algorithm 
can be improved. The quasi-Newton with DFP Scheme algorithm is represented in algorithm 2. 


Algorithm 2 Quasi-Newton with DFP scheme algorithm 

Data: xo € R”, and tolerance, € > 0. Setting Ag = 2, 8 = 0.618 and o = 0.8. 
Result: Minimizer x;,* 

while (del < €) do 


Ag = max{s, s6, 56?,---},8 > 0,8 € (0,1),0 € (0,1) while ((f(ex + Arde) — f(x) < orngf dr ) do 
| Ak = Akb 
end 
£k+1 = Tk + Akdk Compute Bkg+1 by using k=k+1 dk = —Bkg9k 
end 
£k is a minimizer 


4.2. Broyden-Flecher-Godfard-Shanno scheme 
The search direction d;, at k-th iteration is provided by the solution of the analogue of (7) and given 
as (11): 
Bkdk = —9k (11) 


where Bp will approximate the Hessian matrix. Initially, B, 1 =I andg = Vf (£k) are chosen. The matrix 
B will be updated iteratively at each iterations [28]-34]. By letting yk = gk+1 — gk and sk = 7441 — Tk, thus 
the update of By+1 is given by (12): 


YUE — Bess, BE 


Bk+1 = Bk + 
YE sk sT Besk 


(12) 


The following subsection illustrated the quasi-Newton with BFGS scheme algorithm which replace the ex- 
ploitation stage of ABC algorithm. 


4.2.1. The quasi-Newton with BFGS scheme algorithm 

The quasi-Newton with BFGS Scheme algorithm is given. The next subsections describe the re- 
finement of the ABC algorithm by introducing the DFP and BFGS Schemes. In this work, the DFP and 
BFGS schemes are used in the exploitation phase of the ABC algorithm. By doing this, the convergence of 
the original ABC algorithm can be improved. The quasi-Newton with BFGS Scheme algorithm is given in 
algorithm 3. 


Algorithm 3 Quasi-Newton with BFGS scheme algorithm 

Data: xo € R”, and tolerance, € > 0. Setting A, = 2, 8 = 0.618 and o = 0.8. 
Result: Minimizer £* 

while (lds < €) do 


Ag = max{s, 98, 58?,---},8 > 0,8 € (0,1), € (0,1) while (f(x + Arde) — f(x)) < orngfdx) do 
| Ap = Akb 
end 
£k+1 = Tk + Akdk Compute B+, by using k=k+1 dk = —Bkgk 
end 
£k is a minimizer 


4.2.2. The improvement of ABC algorithm with quasi-Newton schemes 

The improvement of ABC algorithm using DFP and BFGS schemes which replaces the exploitation 
process is explained as follows. Furthermore, the hybridization of the ABC algorithm with the DFP scheme 
and the hybridization of the ABC algorithm with the BFGS scheme in the exploitation stage are referred to 
as the ABC-DFP algorithm and the ABC-BFGS algorithm, respectively and represented in algorithm 4. The 
numerical results are presented and given in the following section. The results show that the proposed algorithm 
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is capable in finding the global optimal solution effectively compared to the original ABC algorithm. Moreover, 
the convergence ability of the proposed algorithm is reported in the following section. 


Algorithm 4 ABC Algorithm with quasi-Newton scheme 


Data: Initilization: the population of solution z;j i = 1,--- ,n; 7 = 1,--- , D; trial = 0 is the non improvemnet number of the solution 
x;,; used for abondonment, maximum cycle number, limit = A food source which could not be improved through limit trial and 
abandoned by its employed bee. 

Evaluate the population Set cycle = 1 repeat 

{Produce a new food source population for employed bees} for i=/ to n do 

Produce a new food source v; for employed bees of the food source x; by using Algorithm Apply a greedy selection process 
between v; and x; and select the better one if x; does not improve then 
| trial = trial; + 1 Otherwise trial; + 1 = 0 

end 

end 

Calculate the probability value p; by using {6} for the solutions using fitness values by using {Produce a new food source 

population for onlooker bees} Sett = 0,1 = 0 repeat 

if random < p; then 

{Produce a new v; food source by using Algorithm ph Apply a greedy selection process between v; and x; and select the 
better one if Solution x; does not improve then 
| trial; = trial; + 1 Otherwise trial; +1 =0 t=t+1 


end 

until t = n 

{Determine scout} if max(trial;) > limit then 

Replace x; with a new randomly produces solution by using 


end 
Memorize the best solution achieved so far cycle = cycle + 1 


until {cycle = Maximum cycle Number} 


5. NUMERICAL RESULTS AND DISCUSSION 


The original ABC and the enhanced ABC algorithm with the DFP and BFGS schemes are presented. 
The performance improvement is compared to the original ABC algorithm using eighteen selected global op- 
timization problems taken from [4]. The numerical results comparison is depicted in Table [I] whereby the first 
column represents the name of the test problem while the second together with the third columns represent 
results obtained by using BFGS and DFP schemes in the exploitation part of ABC algorithm. Meanwhile the 
final column depicted the results obtained using the original ABC algorithm. 


Table 1. The performance of enhanced and original ABC algorithms 


Problem Function BFGS DFP ABC Algorithm 
1 Griewank (5 variable) 1.313384 x 1073 1.630763 x 1078 2.934116 x 10710 
2 Sphere 2.055125 x 10-18 2.055125 x 10713 7.855848 x 1076 
3 Rosenbrock 1.828925 x 10-18 8.927977 x 1071? 4.081273 x 107! 

(5 variable) 
4 Rastrigin 6.069244 5.820506 2.448230 x 10? 
(5 variable) 
5 Rastrigin (2 variable) 4.477316 x 1071 3.482357 x 107! 8.809675 x 1071! 
6 Ackley 2.630402 x 1076 2.774587 x 107 2.181409 x 10-7 
7 Beale 3.561118 x 10714 3.846104 x 10714 4.939879 x 10-5 
8 Goldstein-Price 3.000000 3.000000 3.018673 
9 Booth 2.611529 x 10-18 2.526855 x 10-18 1.127488 x 1075 
10 Matyas 4.725471 x 10-15 5.988413 x 10-15 1.684934 x 1075 
11 Lévi 4.526399 x 10712? 1.919452 x 10-1! 5.493683 x 1073 
12 Three-hump camel 1.672050 x 10-18 1.791896 x 10713 3.411217 x 1071 
13 Easom —1.000000 —1.000000 —1.000000 
14 Adjiman —5.002938 —4.998900 —1.001762 x 10! 
15 Bird —1.067645 x 10? = —1.067645 x 10? — 1.067645 x 10? 
16 Bohachevsky 1 4.293244 x 10712 4.742057 x 1071? 1.137368 x 10713 
17 Bohachevsky 2 1.800000 x 1071 1.800000 x 1071 1.800000 x 1071 
18 Bohachevsky 3 3.065742 x 10-18 3.727768 x 10-18 5.237552 x 1076 
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Results obtained proved the proposed algorithm has the ability to find the global optimum solution 
effectively when compared to the original ABC algorithm except for the Griewank and Rastrigin functions. 
Moreover, convergence ability of the proposed algorithm is still better than the original ABC algorithm. Con- 
clusively, the proposed algorithm attained a smaller margin of error in eight out of the eighteen test problems. 


The success of the new approach is because of the Armijo line search beeing used in the enhancement 
process. According to literature, the quasi-Newton method using Armijo line search will hold global convergent 
ability like the steepest descent method. Therefore, the new approach algorithms can perform as great as the 
steepest descent method [15]. 


The performance of all two variants of the new approach which ABC-DFP and ABC-BFGS algorithms 
have been compared to the original ABC algorithm in 10 designated global optimization problems [4]. The 
numerical results showed by the graph of the algorithm to obtain a global optimum solution with the number 
of iterations. Obviously, numerical results indicated that the new approach algorithms are able to obtain more 
accurate a global solution than original ABC algorithm except in Griewank function. However, in all selected 
problems the new approach showed their convergent ability is better than the original ABC algorithm. 


6. CONCLUSION 


This paper attempted to address the issue at exploitation stage for ABC algorithm. The complexities 
and inefficiencies in evaluating second derivatives and difficulty in predicting initial point is tackled during this 
research. The new proposed algorithm explored modification of the exploitation part in ABC algorithm by 
introducing a quasi-Newton algorithm that included DFP and BFGS schemes. All the considering algorithms 
are capable in obtaining the global optimal solution of the global optimization problems. Numerical results 
comparison proven that the new proposed algorithm is capable in getting the optimal global solution effectively 
in the majority of test functions. This finding is significant in terms of enhancing the algorithm to obtain the 
solution associated with global optimization problems. On the other hand, the ability of the original DFP and 
BFGS which used to find local optimum, have been improved by hybridizing with ABC and able to determine 
the global optimum solution in nonconvex multimodal optimization problems. 
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